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ABSTRACT 


An experiment is in progress to measure the effectiveness of 
Ada in the National Aeronautics and Space Administration/ 
Goddard Space Flight Center flight dynamics software devel- 
opment environment. The experiment features the parallel 
development of software in FORTRAN and Ada. The experiment 
organization, objectives, and status are discussed. Experi- 
ences with an Ada training program and data from the devel- 
opment of a 5700-line Ada training exercise are reported* 


INTRODUCTION 


An experiment is underway to assess the effectiveness of Ada 
for flight dynamics software development. This paper is an 
interim report on the experiment, discussing the objectives, 
organization, preliminary results, and plans for completion. 


*Ada is a registered trademark of the U.S, Government (Ada 
Joint Program Office) . 

** Proceedings, Tenth Annual Software Engineering Workshop , 
National Aeronautics and Space Administration, Goddard 
Space Flight Center, December 1985. 

***Author's Address: Computer Sciences Corporation, System 

Sciences Division, 8728 Colesville Road, Silver Spring, 
Maryland 20910. 
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The Ada experiment is planned and administered by the Soft- 
ware Engineering Laboratory (SEL) of the National Aeronau- 
tics and Space Administration's Goddard Space Flight Center 
(NASA/GSFC) . NASA/GSFC and Computer Sciences Corporation 
(CSC) are cosponsors of the experiment. Personnel from all 
three SEL participating organizations (NASA/GSFC, CSC, and 
the University of Maryland) support the experiment. 

TECHNOLOGY ASSESSMENT IN THE SEL 


There is a great deal of optimism concerning Ada's potential 
effect on software development. The SEL seeks to establish 
an empirical basis for understanding Ada's effectiveness in 
a particular environment — namely flight dynamics software 
development at NASA/GSFC. Figure 2* shows some of the char- 
acteristics of this development environment. (Reference 1 
contains a more detailed description.) 

As Figure 2 implies, in seeking to understand the effective- 
ness of Ada, the SEL is approaching this task as it has 
addressed the assessment of other software technologies. 

Some methods that have been demonstrated to be effective in 
other environments have not been effective in the SEL envi- 
ronment. The SEL is therefore cautious about expecting that 
reported experiences with Ada will obtain in the SEL envi- 
ronment. Instead, the SEL seeks to conduct an assessment of 
Ada in its own environment. 

The assessment methods used by the SEL have included con- 
trolled experiments, case studies, and analytical investiga- 
tions. The Ada assessment is referred to as an experiment, 
although it is clearly not a controlled experiment. Iden- 
tifying this effort as an experiment follows the general use 


* All figures are grouped together at the end of the paper. 
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of the word to denote "any action or process undertaken to 
discover something" (Reference 2) . As the later discussion 
will make clear, the Ada experiment is a highly instrumental 
case study of an Ada implementation in parallel with a 
FORTRAN implementation, with both systems developed in re- 
sponse to the same requirements. 

OBJECTIVES 


The primary objective of the experiment (Figure 3) is to 
determine the cost-effectiveness of Ada and its effect on 
the flight dynamics environment. A related objective is to 
assess various methodologies that are related to the use of 
Ada. An initial set of such methodologies includes object^ 
oriented design (Reference 3) , the process abstraction method 
(Reference 4) , and the composite specification model (Refer- 
ence 5) . Additional methodologies will be identified as the 
experiment continues. 

Reusability is an important tactic for cost-effective soft- 
ware development, both in a general sense and in the SEL 
environment. Ada was designed (in part) to facilitate re- 
usability. This experiment seeks to develop approaches for 
reusability when Ada is the implementation language. 

The Space Station is a program of great size, complexity, 
and significance to NASA. Ada has been recommended as the 
language to be used for the development of new software for 
the Space Station. An objective of the Ada experiment is to 
develop measures that may assist in planning for the large- 
scale use of Ada in the Space Station program. Examples of 
such measures are those that relate to size, productivity, 
or reliability in an Ada implementation. 
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Because the experiment is not completed, these objectives 
have not yet been met. However, experiences thus far will 
contribute to addressing the objective of understanding the 
effect of Ada. 


EXPERIMENT PLANNING 

The experiment consists of the parallel development, in 
FORTRAN and Ada, of the attitude dynamics simulator for the 
Gamma Ray Observatory (GRO) (Figure 5) ; which is scheduled 
to be deployed in May 1988. It is worth noting that the 
dynamics simulator is part of the standard complement of 
ground support software planned for the GRO mission. The 
simulator would routinely be developed in FORTRAN alone; 
because of the experiment, it is being developed in Ada as 
well. 

When completed, the system is expected to comprise 
40,000 source lines of (FORTRAN) code, requiring 18 to 
24 months to develop on a VAX-11/780 computer. Each team 
was staffed initially with seven personnel from NASA/GSFC 
and CSC. Each development project is expected to require 8 
to 10 staff-years of effort. 

Three teams have a role in the experiment (Figure 6) : the 

Ada development team; the FORTRAN development team; and an 
experiment study team consisting of NASA/GSFC, CSC, and 
University of Maryland personnel. The study team is respon- 
sible for planning the experiment, collecting data from the 
development teams, and evaluating the progress and results 
of the experiment. The study team will also be able to com- 
pare the software products generated by each team. 

The profiles of the development teams (Figure 7) reveal that 
the Ada team on average is familiar with more programming 
languages and is more experienced than the FORTRAN team. 
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However, the Ada team is less experienced with dynamics sim- 
ulators, the application area of interest. 

Striking differences exist in the relationships of the teams 
to their development tasks (Figure 8) . The FORTRAN team is 
able to reuse some design and code from related systems. 

The Ada team is charged with starting fresh to design a sys- 
tem that can take advantage of Ada-related design approaches. 
For the Ada team, both the development environment and the 
language are new. 

Figure 9 shows the timeline for the Ada experiment with the 
activities of the three teams during the expected 2-year 
duration of the experiment. The timeline shows the FORTRAN 
team to be slightly more than one development phase ahead of 
the Ada team. The shift is due to the training in Ada re- 
quired by the Ada team at the start of the project. The 
FORTRAN team, by contrast, was able to start immediately 
with the requirements analysis activity — the first phase in 
the development process. 

The study team is collecting data on both development teams. 
Figure 10 shows the range of resource, project, and product 
data collected. Wherever possible, routine SEL forms were 
used. However, special Ada versions of two forms--the com- 
ponent origination form and the change report form — were 
developed. The new component form allows the identification 
of an Ada component as a package, task, generic, or subpro- 
gram and further recognizes that a component can be a speci- 
fication or body. The new change form adds a section to 
identify separately any Ada-related errors. 

TRAINING APPROACHES 

A major portion of the 'experiment thus far has been the Ada 
training program, which was planned by the study team, in 
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particular by the University of Maryland personnel. The 
principal training resources (Figure 12) were as follows; 

• Ada language reference manual (LRM) (Reference 6) 

• Ada textbook (Reference 3) 

• Ada videotapes (Reference 7) 

The 27 videotapes were viewed by the team over a 1-week pe- 
riod. A University of Maryland graduate student, experienced 
in Ada, was available to direct the training--that is, to 
plan the schedule of tape viewing, answer questions about 
Ada material, stop the tapes to clarify the material, lead 
the discussion between tapes, and assign reading and small 
coding assignments. Two sets of diskettes for use on per- 
sonal computers were available to the team to supplement the 
videotaped instructions. Lectures on Ada-related design 
methods--the state-machine abstraction and process abstrac- 
tion method (Reference 4) --were presented to the team. 

A principal component of the Ada training program was the 
design and implementation in Ada of a practice problem. The 
purpose of this training exercise was to enable the team to 
apply what it had been taught about Ada and to begin working 
together as a team. 

Figure 13 shows the coverage of topics by the training ele- 
ments. The textbook and the training exercise covered all 
three training topics: the Ada language itself, software 

engineering with Ada, and Ada-related design methods. 

Experience with Ada training led to several recommendations 
for future sessions (Figure 14) . Consistent with several 
other published recommendations (e.g.. Reference 3), the 
appropriate emphasis should be on software engineering with 
Ada and not simply the language syntax and semantics. The 
methods and resources used in training the Ada team-- 
videotapes, class discussion, and a practice problem--were 
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effective. Additional hands-on experience with the Ada com- 
piler (in addition to work on the practice problem) is also 
beneficial. 

Two months of full-time training are recommended for each 
staff member. After this period, the staff member would be 
able to join a development team and begin contributing. 
Ideally, this first assignment as a developer should be 
carefully chosen and closely monitored by a more senior de- 
veloper. Reference 8 contains a more thorough assessment of 
Ada training methods and more detailed recommendations for 
the design of future Ada training programs. 

DATA FROM THE ADA TRAINING EXERCISE 


The training exercise (or practice problem) emerged as the 
single most valuable element of Ada training. It also pro- 
vided the study team with an opportunity to practice moni- 
toring a small Ada project. 

The exercise was to design and develop an electronic message 
system (EMS) that allows users to send and receive elec- 
tronic mail and to manage groups of users (Figure 16) . EMS 
has been used as a student programming project at the 
University of Maryland, where it was implemented in the SIMPL 
language, requiring typically 1000 to 2000 lines of code. 

For the Ada team, EMS was a chance to practice object- 
oriented design as well as to experiment with Ada. The 
study team could try out the data collection system and 
begin measuring a small Ada development. 

The completed EMS system in Ada comprised 5730 lines of code 
(Figure 17) , much larger than the student projects in SIMPL. 
An analysis is currently underway to compare the functional- 
ity of the Ada and SIMPL versions. It is already clear that 
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the Ada version has a much more extensive user interface and 
help facility. Also, the 5730 source lines contained only 
1402 executable statements. The drop from source lines to 
executable statements is more severe than in SEL FORTRAN 
systems, where reductions of only 2 to 1 are typical. 

Developing EMS required 1906 staff-hours (including 570 hours 
of training) , A productivity/cost measure frequently used 
in the SEL is the number of hours per thousand executable 
statements. Figure 17 shows the cost of EMS development to 
be greater than the average cost of developing FORTRAN sys- 
tems. Of course, the EMS example in Ada represents only a 
single data point whereas the FORTRAN cost data are taken 
from hundreds of FORTRAN modules in the SEL data base. 

It is wise not to rely too heavily on the EMS data as an 
indicator of future Ada projects. There are several sound 
reasons why the costs could be higher or lower than those 
experienced with EMS. 

Costs could be higher in the future because of the following: 

• EMS was developed by a highly motivated staff eager 
to apply Ada. As the use of Ada becomes more routine, the 
staff may not be as motivated by the novelty of using a new 
language in an experimental setting. 

• EMS had no documentation requirements, unlike typi- 
cal SEL projects. 

• EMS did not involve tasking. 

• The application domain of EMS (electronic mail) was 
easier to understand than the flight dynamics area. As a 
result, the EMS effort in requirements analysis and accept- 
ance testing was proportionally less than it would be for 
flight dynamics projects. 
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Costs of the Ada development may actually be lower than sug- 
gested by EMS because of the following: 

• The staff will be better trained. Recall that EMS 
was a training exercise; teams in the future will be more 
experienced in Ada. 

• The Ada team (with seven people) was too large for 
the EMS assignment. The size of the team was driven by the 
scope of the GRO dynamics simulator development. The cost 
of EMS would likely have been less if the team were smaller 
(approximately three people) . 

• The Ada development environment for EMS was not 
only new but also highly unstable. Only unvalidated Ada 
compilers were available when coding of EMS began. The team 
progressed through versions 1.3, 1.5, and 2.1 of the Tele- 
soft compiler before the DEC Ada compiler arrived. 

Figure 17 shows that the error rate for EMS was lower than 
that of FORTRAN systems in the SEL data base. Once again, 
this result should not necessarily be attributed to the use 
of Ada on EMS. The FORTRAN systems are much more complex, 
and the testing requirements in the flight dynamics area are 
much more rigorous than for EMS. 

Figure 18 shows the distribution of effort among design, 
code, and test for EMS and typical FORTRAN systems. Whereas 
the relative effort for the three activities is roughly 
equivalent for FORTRAN systems, 60 percent of the EMS Ada 
effort was spent on design. Of course, the use of Ada 
raises the question of redefining the cutoff between design 
and code activities. If Ada is used as a process design 
language (PDL) , the design activity can include the delivery 
of a design document of compiled specifications, Ada defini- 
tions of types, and Ada PDL. In such cases, it may be 
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understandable that more effort is spent on "design" activ- 
ity, with proportionally less effort on "code." Again, the 
more substantial testing requirements for FORTRAN flight 
dynamics systems may explain the difference in relative 
effort devoted to testing EMS versus typical FORTRAN systems. 

The profile of the EMS code in Figure 19 reveals that the 
EMS Ada modules were smaller on average. The lower percent- 
age of lines of EMS that are blank or comment (39 percent 
versus 51 percent) may be due to the greater self -descript ion 
possible with Ada object names and types. 


STATUS AND OBSERVATIONS 


Figure 21 revisits the experiment timeline to show the actual 
activity to date. The activity profiles of the two develop- 
ment teams confirm that progress is being made according to 
plan. 

With the Ada experiment not yet complete, no definitive 
statements can be made on the effectiveness of Ada in the 
SEL environment* Nevertheless, Ada's influence is being 
felt on personnel issues, software products, the development 
environment, and the software development process (Fig- 
ure 22) . 

The clearest observations relate to the activity that has 
dominated the early phases of the experiment — training . The 
need for effective training is real and should be included 
explicitly in Ada development plans. Training will occur 
whether or not it is scheduled; wise managers will plan for 
it. Two months of full-time training appears to be the 
right amount. The training exercise emerged as an extremely 
effective method and is strongly recommended. 
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The use of Ada led to a larger product than the student ver- 
sions of EMS in SIMPL. It is premature to state whether Ada 
products will continue to be larger. EMS did demonstrate 
that many more design relations are expressible in Ada. The 
use of Ada will likely lead to changes in recommended inter- 
mediate products, for example, at design reviews. Current 
recommendations are oriented to FORTRAN implementations, so 
the design products highlight the invocation structure of 
the code. Ada design products can express other relations 
in addition to invocation--for example, the "uses" relation, 
exception handling, and the management of the name space. 

The use of Ada has not degraded the performance of the de- 
velopment environment. Stress test are now in progress, but 
the early indications are that the use of the DEC Ada Com- 
pilation System (ACS) is not adversely affecting the per- 
formance of the system. Both compilation time and execution 
time appear to be within acceptable limits, although more 
complete testing is being performed. 

The most important tool is a validated compiler. The DEC 
ACS has demonstrated that it is a production-quality system. 
Although other Ada support tools may be used by the team in 
the future, the DEC ACS has been adequate by itself to sup- 
port development. The library management facility built 
into the ACS has been especially helpful. 

Although such compulsions may appear less than daring, the 
Ada experiment has demonstrated that Ada is learnable and 
that an Ada project is measurable. The results thus far 
lead the study team to be optimistic that they will be able 
to meet their experimental objectives and establish an 
empirical basis for understanding the effect of Ada in the 
flight dynamics software development environment. 
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FIGURE 6 

EXPERIMENT ORGANIZATION 
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ADA TRAINING EXERCISE 
ELECTRONIC MESSAGE SYSTEM (EMS) 
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ADA EMS PROJECT SUMMARY 
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EFFORT DISTRIBUTION BY ACTIVITY TYPE 
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PROFILE OF ADA EMS CODE 
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EXPERIMENT PLANNING 
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OBSERVATIONS ON THE IMPACT OF ADA 
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